DOCUMENT RESUME 



ED 339 738 



TM 017 660 



AUTHOR 
TITLE 



INSTITUTION 
SPONS AGENCY 

REPORT NO 
PUB DATE 
CONTRACT 
NOTE 

AVAILABLE FROM 

PUB TYPE 

EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Miller, Timothy R. 

Empirical Estimation of standard Errors of 
Compensatory MIRT Model Parameters Obtained from the 
NOHARM Estimation Program. ACT Research Report 
Series. 

American Coll. Testing Program, Iowa City, Iowa. 
Office of Naval Research, Arlington, VA. Cognitive 
and Neural Sciences Div. 
ONR-91-2 
Aug 91 

N00014-89-J-1908 
38p. 

ACT Research Report Series, P.O. Box 168, Iowa City, 
lA 52243. 

Reports - Research/Technical (143) 
MF01/PC02 Plus Postage. 

College Entrance Examinations; Comparative Analysis; 
*Coir.puter Simulation; Equations (Mathematics); *Error 
of Measurement; *Estimation (Mathematics); Higher 
Education; High Schools; *High School Students; item 
Bias; *Item Response Theory; Mathematical Models; 
Mathematics Tests; Standardized Tests 
Empirical Research; Item Parameters; Multidimensional 
Models; *NOHARM Computer Program; Preliminary 
American College Test Plus 



ABSTRACT 

Two studies were carried out to evaluate the quality 
of multidimensional item response theory (MIRT) model parameter 
estimates obtained from the computer program NOHARM. The purpose of 
the first study was to compute empirical estimates of the standard 
errors of the parameters. In addition, the parameter estimates were 
evaluated for bias and the effects of using different starting values 
and anchor items. Real data from a 1987 national administration of a 
form of the Preliminary American college Test (P-ACT) mathematics 
test with a sample of 30,000 cases and 10 replication samples of 
2,000 students each. The second study was included to compare the 
performance of NOHARM with the findings of an earlier simulation 
study that evaluated other MIRT estimation programs. The results are 
generally good, with fairly small standard errors for most parameter 
estimates and little indication of bias. Although the estimation 
procedure appeared to be robust under different starting values,- the 
specific choice of items used to anchor the solution appears to have 
important effects on the magnitude of the estimated standard errors. 
The comparison of NOHARM with other programs was very favorable and 
supports the use of NOHARM for practical MIRT applications. Eight 
references are listed, and 10 tables present study data. (SLD) 
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Abstract 

Two studies were carried out to evaluate the quality of multidimensional item 
response theory (MIRT) model parameter estimates obtained from the computer 
program NOHARM. The purpose of the first study was to compute empirical estimates 
of the standard errors of the parameters. In addition, the parameter estimates were 
evaluated for bias and the effects of using different starting values and anchor items. 
The second study was included to compare the performance of NOHARM with the 
findings of an earlier simulation study which evaluated other MIRT estimation programs. 
Results were generally good, with fairly small standard errors for most parameter 
estimates and little indication of bias. Although the estim.ation procedure appeared to 
be robust under different starting values, the specific choice of items used to anchor the 
solution appears to have important effects on the magnitude of the estimated standard 
errors. The comparison of NOHARM with other programs was very favorable and 
supports the use of NOHARM for practical MIRT applications. 
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Empirical Estimation of Standard Errors of Compensatoiy MIRT Model Parameters 
Obtained from the NOHARM Estimation Program 

Introduction 

The practical utility of multidimensional item response theory (MIRT) depends 
upon the ability to obtain reasonably accurate parameter estimates. Several estimation 
programs are currently available, including MIRTE (Carlson, 1987) and MULTIDIM 
(McKinley, 1987) which were developed specifically as MIRT programs, TESTFACT 
(Wilson, Wood and Gibbons, 1984) which is a full information item factor analysis 
program that can be used to obtain MIRT parameter estimates, and NOHARM (Fraser, 
1986) a general program fcr fitting unidimensional and multidimensional normal ogive 
models by a least squares procedure. An earlier simulation study (Ackerman, 1988) 
compared MIRTE, MULTIDIM and TESTFACT along several criteria and found 
MULTIDIM and TESTFACT to be far superior to MIRTE, with TESTFACT 
performing the best overall under the conditions of that study. 

In this study, NOHARivi is evaluated for its accuracy and usefulness as a MIRT 
program. The main question is whether the estimates provided by NOHARM are 
sufficiently accurate for practical applications. Since NOHARM employs a least squares 
procedure, standard errors are not directly availab' . and must be established empirically. 
The purpose of this study is to estimate, through approximation of the sampling 
distribution by repeated sampling, the standard errors of the parameter estimates 
provided by NOHARM. 

In addition to estimating standard errors, this research will evaluate the estimates 
for bias and the effects of using different starting values and different anchor items to fix 
the solution. Finally, the performance of NOHARM is compared with the other 
programs mentioned above. The assessments of standard errors, bias, and robustness 
will involve analyses of real datasets. The comparison with other programs will be 
accomplished through a simulation identical to that used by Ackerman (1988). 
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The NOHARM Model and Procedures 

NOHARM (Normal Ogive Harmonic Analysis Robust Method) is a program for 
fitting unidimensional and multidimensional normal ogive item response models. The 
generalized multidimensional normal ogive model is given as 

/»(y^=l |6;=c,4.(l-c,)*y^+«^e;. (1) 

where P(x^^-\\ai, d^, 6j) is the probability in an m-dimensional space of a correct 
response to item / by person a; is an m-dimensional vector of item discrimination 
parameters, dj is a scalar parameter related to item difficulty, 6j is an m-dimensional 
vector of latent abilities, q is a pseudo-guessing parameter, and t is the normal 
distribution function. 

The model is fit by an ordinary least squares procedure which seeks to minimize 
the squared differences between the sample and estimated bivariate proportions correct. 
A four term polynomial series is used to approximate the model given by equation (1), 
and the estimated bivariate proportions correct are derived from this approximation, 
allowing the minimization with respect to the model parameters d, a, and Zq. The 
vector c is not estimated but is treated as fixed. The function to be minimized is a least 
squares function and is minimized using a conjugate gradients minimization algorithm. 

To run the program, the vector c must be supplied by the user. This can be a null 
vector, in which case a multidimensional extension of the two-parameter model is 
invoked, a vector of a priori values supplied by the user, or a vector of estimates 
obtained from some other program such as BILOG (1989). The user may specify either 
an exploratory or confirmatory analysis. In either case, starting values for the parameters 
to be estimated may be supplied by NOHARM or the user. The default starting values 
are .5 for the a-parameters and .1 for any off-diagonal elements of the correlation 
matrix that may be estimated in a confirmatory analysis. In general, the solution is 
anchored by fixing items to load only on certain dimensions. If the analysis is two 
dimensional, a single item will be fixed to load only on the first dimension. For a three 
dimensional analysis, a second item is fixed to load only on the first two dimensions, and 
so on. If the analysis is exploratory the pattern matrix is set such that the first m-l items 



are fixed in this manner. In a confirmatory analysis the user may specify which items are 
used to anchor the solution. Also, in a confinnatory analysis, the user may allow for 
correlated thetas while in the exploratory mode the analysis is orthogonal. For further 
details on running NOHARM the reader is referred to Fraser (1986). 

The program estimates the d-parameters and a-parameters, and, when 
appropriate, the off-diagonal elements of Ze- Other output includes the residual 
covariances of the items and the root mea^ square of these values. The program also 
provides the common factor model parameterization of the normal ogive model 
parameters, and, when the analyses are exploratory, provides Varimax and Promax 
rotations of the pattern matrix. 

In addition to the parameters of the multidimensional normal ogive, this study will 
compute and evaluate indices proposed by Reckase (1985, 1986) for multidimensional 
item difficulty (MDIFF) and multidimensional item discrimination (MDISC). MDIFF 
consists of a set of statistics that describes item difficulty as the direction from the origin 
in the multidimensional space in which the item provides the most information and the 
signed distance in that direction to the most informative point on the item response 
surface. For a given item, the direction cosines of MDIFF are given by 



where the ajj^ are elements of the vector B; given in equation 1, The distance component 
of MDIFF is given by 



COS«it= 




,2 2 



(2) 



(m \i 



(3) 



where dj is the item difficulty index given in equation 1. 
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MDISC indicates item discrimination in the MDIFF direction and is given as, 



MDISC- 



E4 



\ (4) 



To summarize, the parameters of interest in this study were: 

1. a - the (i X m) matrix of NOHARM estimated item discriminations 

2. d - the (i x 1) vector of NOHARM estimated item difficulties 

3. MDISC - the (i x 1) vector of multidimensional item discriminations 

4. a - the (i x m) matrix of angles obtained from the cos a components of 
MDIFF 

5. D - the (i x 1) vector of distance components of MDIFF 

Two separate studies are reported. The first involves real data and was designed 
to establish empirical estimates of standard errors, assess bias, and evaluate the effects of 
using different starting values and anchor items. The second study consisted of a 
simulation intended to compare NOHARM with other estimation programs. Following 
the design of the Ackerman (1988) study, the focus was on the ability to reproduce (^'^ta 
using NOHARM estimated item parameters. 



^ylethod 

Real Data Analyses 

Data. The data used in this study were obtained from a 1987 national 
administration of a form of the P-ACT* mathematics test. This test is given primarily to 
high school sophomores and consists of 40 multiple-choice items measuring achievement 
in the content areas of pre-algebra, algebra, plane geometry and coordinate geometry. A 
"population" sample of 30,000 cases was selected at random from a total administration 
sample of approximately 140,000 examinees. Ten replication samples of n=2000 each 
were then selected at random and with replacement from the population sample. 

Analyses. Earlier factor analyses of several PACT datasets had suggested three 
factors, interpreted as a geometry factor, an algebraic symbol manipulation factor, and a 
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word problems factor. A preliminary NOHARM analysis of the 30000 case sample was 
carried out in three dimensions to confirm this structure and to assess how well this 
model would fit the "population" data, an important pre-requisite for the subsequent 
analyses. Results indicated a very good fit, with a root mean squared residual (RMSR) 
product moment of .003. Therefore, product moment matrices for each of the 10 
samples were also fit by a three-dimensional model. Estimates of the Cj-parameters were 
obtained from a unidimensional analysis using BILOG (1989) and were input as fixed 
values for the NOHARM analyses. Initially, default settings were employed, so that the 
first two items were used to anchor the solution (see earlier discussion), starting values 
were .5 for the a estimates, and the solutions were orthogonal. Additional analyses were 
carried out to assess the effects of using different starting values and different anchor 
items. For questions related to starting values, three additional analyses were carried out 
on the population sample using starting values of .3, .8 and 1.5. To assess the effects of 
using different anchor items, the ten replication samples were re-run using two different 
sets of two anchor items. 

As stated earlier, the main interest in this study was in obtaining empirical 
estimates of the standard errors of the parameters. This was accomplished by computing 
the standard deviations of the parameter estimates for the 10 replications. This was 
done for both the NOHARM model parameter estimates as well as the MIRT statistics. 
In addition, an estimate of bias was computed for each parameter as the average of the 
difference between each of the ten estimates of that parameter and the "population" 
value. For the follow-up studies pertaining to starting values, the d and a estimates were 
averaged over items and these averages were compared across the different analyses. 
Also, correlations were obtained for each set of 40 parameter estimates across the 
different starting value conditions. For the analyses involving different anchor items, the 
main concern was whether the arbitrary use of the first m-1 items as anchors would lead 
to unnecessarily high standard errors. Therefore, for these analyses the standard errors 
were re-computed for the different configurations and compared with those obtained 
under the default conditions. 
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Analysis of Simulated Data 

Data. Data for the simulation were generated from a multidimensional two- 
parameter logistic (M2PL) model using bivariate normal theta distributions and item 
parameters from an earlier study (Ackerman, 1988). These parameters, given in Table 1, 
were selected to provide uniform information over the ability continuum. Fifty items and 
two dimensions were used in the simulation. Two data sets of n=2000 were generated, 
one with rgjg2=0.0 and the other with rgjg2=0.5. 



Insert Table 1 about here 



Analyses. The purpose of the simulation study was to investigate how well input 
data could be reproduced using NOHARM estimated item parameters. NOHAf^M was 
used 10 obtain two dimensional solutions for each of the datasets. Default settings were 
employed for both an.ilyses, with the c-parameters fixed to zero to create a 
multidimensional extension of the 2-parameter model, ./i order to compare the results 
of this study Wkth those of the earlier study, estimates of ability were needed. Since 
NOHARM does not provide such estimates, a program was written to compute expected 
a posterior (EA?) means for each examinee. The choice to use EAP scores was made to 
provide the most direct comparison with TESTFACT. 

For each person and item, a standardized residual was computed as 

^\=7^ (5) 

where yjj is a 0/1 score on item / for person ;, and p,^ is the expected probability of a 
correct response on item / for person ; computed from equation 1. The focus of the 
evaluation was on the moments of the distribution of the residuals for each item and on 
the average of the means and standard deviations of these values over items. The mean 
residuals (both for individual items and overall) will serve primarily to provide a check 

12 
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on the accuracy of the estimation procedure and should be very near zero if the program 
is functioning properly and providing unbiased estimates. However, assessment of bias 
alone is not sufficient to address the practical utility of the procedure, since a procedure 
may be unbiased but have such high variance that it is practically useless. A better 
indication of the overall quality of the procedure will be provided by the standard 
deviations of the fitted residuals. 

Results 

Real Data Analyses 

Tables 2 and 3 contain the means, average biases and standard deviations 
(empirical standard errors) for the NOHARM and MIRT parameter estimates, 
respectively. The last row in each table gives the means of these values over items. 
From Table 2 it can be seen that the overall average of the empirical standard errors for 
d is .15 and ranges from .12 to .15 for the a's. For the MIRT statistics, the average 
standard errors are .17 for MDISC, .09 for D, and range from 5.76 degrees to 7.04 
degrees for the a's. Inspection of the standard errors at the item level indicates that 
most of the parameters were reasonably well estimated. There were however some 
notable exceptions. For example, the estimates of d, a^, and MDISC for item 1 were 
extremely unstable, indicating a possible problem in using that item to anchor the first 
axis. There was also a tendency for the d and MDISC estimates to be less stable for the 
more difficult items (indicated by large negative values for d). On the other hand, D, 
the distance component of MDIFF seems to have been generally well estimated. For the 
fljij, there appears to be a tendency for the estimation to become less stable in the second 
and third dimensions. For the a^^^ this occurred only for the third dimension. 

Overall, there seems to be little important bias occurring. As with the standard 
errors, some exceptions can be found at the individual item level. Note in particular that 
d, aj and MDISC for Item 1 were apparently quite far off the value obtained in the 
analysis of the large sample, again suggesting a possible problem in using this item to 
anchor the solutions. 
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Additional Analyses: Starting Values and Anchor Items 

The follow-up analyses were intended to address two questions: (1) would it be 
possible to reduce the standard errors of the estimates by a better choice of anchor items 
and (2), how sensitive is the analysis to the choice of starting values for the a- and d- 
parameters? 

There were two reasons for the concern over the choice of anchor items. First, in 
many tests, including the PACT+, the items are ordered by difficulty so that the first 
items are easier ai?d generally less discriminating. The question was whether the use of 
items with relatively low discriminations as anchor items would lead to less stable 
solutions and poorer estimates overall than might be obtained by using items with better 
discrimination. The second concern stemmed from the fact that in solutions involving 
m > 2 dimensions, the first w-1 items are chosen arbitrarily by NOHARM as the anchor 
items. Alternatively, it would seem advantageous to use items to anchor different 
dimensions that were somehow known to measure different dimensions. 

To address these questions the analyses were re-run on the ten replication 
samples using two different sets of anchor items. The first set was chosen on purely 
statistical grounds: two items (items 18 and 24) were chosen that were found to have 
average values of difficulty (d) and multidimensional discrimination (MDISC) in the 
default analyses. The other set of items was chosen on substantive grounds: the results 
of a previous factor analysis were used to identify two items (items 3 and 32) that loaded 
on fairly distinct dimensions. As in the previous study, empirical standard errors were 
computed as the standard deviations of the parameter estimates over the ten 
replications. 

Tables 4 and 5 contain the average of the empirical standard errors over items for 
the original analyses using NOHARM defaults and the two additional sets of analyses. 
Contrary to expectations, the use of different anchor items not only failed to improve the 
standard errors but actually caused them to increase, in some cases substantially. 
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Although the standard errors of item 1 were reduced to some extent, the standard errors 
of one of the new anchor items increased. For example, in the 18/24 analysis, the 
standard error of a^ for item 1 was .34, down considerably from its value of .60 in the 
default analysis. However the standard errors of a^ for item 18 in the 18/24 analysis 
inflated from .12 to .82. Similar results were obtained for the other parameters of item 
18 in this analysis and for item 32 in the 3/32 analysis. Thus it seems that the problem 
is not so much which items are fixed but rather the method itself which leads to larger 
standard errors for the fixed items. Nevertheless, it is not altogether clear why selecting 
items on substantive grounds led to increased standard errors overall. Further research 
is needed to clarify these findings. 



Insert Tables 4 & 5 about here 



The results of the analyses run under different starting values are summarized in 
Tables 6 and 7. Recall that three additional analyses were carried out on the population 
sample of n= 30000 using starting values of .3, .8 and 1.5. Table 6 gives the means and 
standard deviations of the NOHARM parameter estimates for these analyses along with 
those from the default analyses. The correlations between the estimates for each of the 
starting value conditions are given in Table 7. 

The results given in Table 6 indicate that varying the starting values had some 
impact, although the effects are not large and are somewhat inconsistent. Increasing the 
starting values led to a decrease in the levels of parameter estimates, with the exception 
of aj under starting values of 1.5. There was also a tendency for the variability of the 
estimates to decrease with larger starting values, although again the trends were not 
consistent. Moreover, since the standard deviations reported in Table 6 are not 
estimates of standard errors, it is difficult to make valuative judgements regarding 
increased or decreased variability. 

The correlations reported in Table 7 reveal a relationship between the degree of 
correspondence between the a^ estimates obtained from different starting values and the 



closeness of those starting values. In general, the greater the disparity between starting 
values, the lower the correspondence between estimates. This trend was not observed 
for the d estimates. 
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Analyses of Simulated Data 

Tables 8 and 9 contain the summary statistics of the residual analyses for the 
'"6162=0.0 data (Dataset 1) and the reie2=0.5 data (Dataset 2), respectively. The results 
indicate that NOHARM performed well in terms of being able to reproduce the data 
with little or no bias on average. At the item level, the mean residuals were less than 
.01 in absolute value for 42 of 50 items in Dataset 1 and 38 of 50 items in Dataset 2. 
The overall mean residual was .001 for Dataset 1 and .000 for Dataset 2. While it is 
apparent that some extreme values occurred, the magnitudes of the standard deviations 
of the residuals suggest that the estimated probabilities of correct response were 
reasonably well behaved. For comparative purposes. Table 10 presents the overall mean 
and standard deviation of the residuals obtained form the NOHARM analyses along with 
those obtained for the other estimation programs evaluated in the Ackerman (1988) 
study. It is apparent that NOHARM and TESTFACT were equally effective in 
reproducing the data as reflected by the lack of average bias in the residuals. Both 
programs also appear to be roughly equivalent in terms of the variance of the residuals. 



Insert Tables 8, 9 & 10 about here 
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Summary and Conclusions 
The parameter estimates provided by NOHARM, along with MIRT item statistics 
computed from those estimates, were evaluated in terms of their estimated standard 
errors, bias relative to population values, and robustness under different starting 
configurations. In addition, a simulation was carried out to permit comparisons with an 
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earlier study that evaluated and compared several other estimation programs. 

For most of the items the estimated standard errors of the parameter estimates 
seemed to be reasonably small, and there was little indication of important bias in the 
estimation. Overall, D, the distance component of MDIFF was the most stable 
parameter, while the and estimates were the least stable. Also, the estimation 
procedure used by NOHARM seems fairly robust to different starting values. Somewhat 
surprisingly, attempts to improve the standard errors by using different anchor items 
were unsuccessful. It is not clear why the arbitrary use of the first m-1 items as anchors 
of an m-dimensional solution led to lower standard errors than did the use of items 
selected on statistical or substantive grounds. It does, however, appear that regardless of 
which items are chosen as anchors, the parameters for at least one of them will be 
poorly estimated. Further research is needed to clarify these findings. 

Although it was necessary in the simulation study to employ an external program 
to obtain the needed ability estimates from the NOHARM analysis, the results 
nevertheless indicated that both the marginal maximum Hkelihood algorithm used by 
TESTFACT and the least squares algorithm used by NOHARM were equally effective at 
reproducing data under well-fitting model conditions. Together the findings of this study 
support the use of NOHARM in practical MiRT applications. 
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Table 1 

Uniform Information Item Set 



Item 



No. 


^1 

1 




D 


d 


MDISC 


a 


1 


1.351 


0.270 


-2.499 


3.442 


1.377 


ii3i;i 


2 


0.653 


1.136 


0.008 


-0.011 


1.311 




3 


1.365 


0.027 


-0.791 


1.080 


1366 


i 151 


4 


0.298 


1.450 


2.482 


-3675 


1 481 


78 386 


5 


1.391 


1.171 


2.495 




1 818 


4n08Q 


6 


1.828 


0.000 


0.470 


-0.860 


1 828 


0 001 


7 


1.796 


0.011 


-0.985 


1J69 


1.796 


0 365 


8 


1.474 


0.017 


2.000 


-2 948 


1.474 


0 644 


9 


0.012 


1.422 


-1.500 


-0.823 


1.422 


89 


10 


0.153 


1336 


2.491 


-3.351 


1345 


83 464 


11 


1.326 


0.286 


2.072 


-2.810 


1356 


12 151 

±JU»X*JX 


12 


1.678 


0.222 


-0.096 


0 163 


1 693 


7 S41 


13 


1.424 


0.001 


-2.498 


3.557 


1.424 


0 042 


14 


0.117 


1.808 


0.869 


-1J74 


1.811 


86 289 


15 


0.176 


1.294 


-0.441 


0.576 


1306 


82 249 


16 


1.414 


0.040 


-2.223 


3.145 


1.415 


1 612 


17 
1/ 




U.UUU 


2.390 


-3.227 


1.3.50 


0.000 


18 


0.236 


1.743 


-2.039 


3.586 


1.759 


82.276 


19 


1.109 


0.839 


■0.240 


0333 


1.390 


37.114 


20 


0.000 


1.438 


1.306 


-1.879 


1.438 


89.999 


21 


0.011 


1.522 


1.747 


-2.660 


1.522 


89.576 


22 


1.399 


0.063 


1.939 


-2.717 


1.401 


2.578 


23 


0.351 


1376 


-0.251 


0.356 


1.420 


75.694 


24 


0.000 


1.568 


1.358 


-2.129 


1.568 


89.990 


25 


0.093 


1.377 


2.384 


-3.290 


1380 


86.131 



(Table continues) 
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Item 



No. 




«2 


D 


d 


MDISC 


a 


26 


0.206 


1.481 


-liOO 


-1.151 


1.495 




27 


1.545 


0.430 


0.894 


-1.434 


1.604 


15.551 


28 


0.404 


1.338 


-2363 


3302 


1397 


73.199 


29 


0.811 


1J22 


-0.934 


1.611 


1.725 


61.944 


30 


1.459 


0.133 


2.047 


-3.000 


1.465 


5.192 


31 


0.606 


2.123 


-2.221 


4.903 


2.208 


74.064 


32 


1.375 


0.002 


2.000 


-2.750 


1375 


0.081 


33 


0.093 


1.640 


-1.975 


3.244 


1.642 


86.739 


34 


0.158 


ISM 


2.500 


-3.781 


1.512 


83.998 


35 


0.000 


1343 


2336 


-3.137 


1343 


90.000 


36 


1.45J 


0.288 


-0.217 


0320 


1.480 


11.241 


37 


1.893 


0.117 


-2.428 


4.604 


1.896 


3.546 


38 


0.026 


1385 


-1.168 


1.617 


1385 


88.909 


39 


0.395 


1351 


0.055 


-0.077 


1.408 


73.712 


40 


2.168 


0.006 


-0.712 


1344 


2.168 


0.150 


41 


0.057 


1355 


1.565 


-2.122 


1.356 


87.603 


42 


0.685 


1.276 


-0.861 


1.246 


1.448 


61.772 


43 


0.064 


1.471 


2.492 


-3.669 


1.472 


87.495 


44 


1.273 


0.815 


2.488 


-3.759 


1.511 


32.622 


45 


0.439 


1.413 


-1.407 


2.082 


1.479 


72.727 


46 


1.451 


0.266 


0.981 


-1.448 


1.475 


10.391 


47 


0.077 


1.425 


-0341 


0.486 


1.427 


86.894 


48 


1.318 


0.036 


-2.393 


3.154 


1318 


1.560 


49 


1.409 


0.000 


-2.500 


3.522 


1.409 


0.009 


50 


1.402 


0.000 


0.401 


-0.563 


1.402 


0.000 
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Table 2 

Means, Average Bias and Empirical Standard Errors ofNOHARM Parameter Estimates 



Item 




d 






ai 
"1 






a^ 










Mean 


Bias 


SD 


Mean 


Bias 


SD 


Mean 


Bias 


SD 


Mean 


Bias 


SD 


1 


4.38 


-.52 


.73 


3.27 


- 51 


.60 


00 


00 


00 


no 


00 


nn 

.UU 


2 


1.33 


.02 


.03 


.52 


.03 


08 


SO 


01 

.V/X 


09 


no 


no 


nn 

.UU 


3 


.77 


.02 


.04 


.49 


-.09 


05 


19 

• x^ 


-03 


09 


15 




in 

.lU 


4 


1.09 


.06 


.06 


.89 


-.08 


.14 


49 


-03 


09 


45 


00 


n6 

.UU 


5 


.51 


.01 


.06 


.43 


-.01 


08 


10 

t XV/ 


00 


07 


.AO 


- ni 


11 

.11 


6 


.21 


.03 


.12 


.62 


-.08 


.08 


47 


-04 


08 


41 

.~x 


ns 


n7 

.u / 


7 


.84 


.01 


.05 


75 


-.06 


07 




-01 

.V/X 


08 




ni 


n7 

.u / 


8 


1.31 


-.03 


.09 


1.16 


- 10 

. XV/ 


14 


09 


ns 


09 


2Q 


. ni 


11 

.11 


9 


.85 


-.01 


.06 


.81 


- 15 

•1.%/ 


10 

.XV/ 


46 


-04 


11 

.XX 






1 1 
.11 


10 


.67 


.02 


.06 


.97 


- 10 


07 




-07 


11 






.UO 


11 


1.31 


-.01 


05 


87 


. 09 


07 
.1/ / 


17 


on 


11 




ni 


no 
.Uo 


12 


1.07 


-.04 


.06 


85 


05 


.09 


46 


04 


05 




- ns 


no 


13 


.67 


.03 


.09 


1.30 




.12 


09 


01 

.V/X 


13 

.x^ 


HQ 


.xo 




14 


-.39 


.03 


.15 


.73 


-.17 


15 

•1.%/ 


.70 


-08 


17 

.X / 




no 


17 

.1 / 


15 


-.52 


-.15 


.22 


.59 


-.03 


.14 


.69 


.06 


15 




18 
.xo 


10 
.17 


16 


-.48 


-.16 


.31 


.25 


-.04 


.16 


.74 


-.02 


26 


22 


13 


32 


17 


.99 


.01 


.09 


1.43 


-.04 


.15 


.16 


-03 


17 

.X f 


94 


- 17 
.1 / 




18 


-.32 


.02 


.15 


.65 


-.05 


.12 


60 


-.07 


15 


50 


ni 


nn 

.uo 


19 


-.07 


.00 


.09 


.50 


-.06 


06 


36 


- 03 


06 


31 

.<JX 


ns 

.UJ 


n7 

.u / 


20 


-.16 


.04 


.09 


.84 


-.12 


.12 


.80 


- 10 

.XV/ 


13 


60 


11 
.11 


11 

.1.3 


21 


.16 


-.09 


.06 


.79 


.17 


.14 

. x~ 


95 


18 


22 


43 


-no 


91 

.^1 


22 


-.11 


-.10 


.10 


.60 


.11 


.08 


95 


21 

.^x 


11 
.XX 


34 


ni 

.Ul 


1 s 
.IJ 


23 


-.23 


-.15 


.16 


.38 


.05 


.14 


.90 


06 


26 


33 


11 

.11 


1^ 
.lu 


24 


.20 


.02 


.06 


.79 


.01 


06 


.27 


-02 


07 
.1/ / 


5H 


ni 

■Ul 


nn 

.uo 


25 


-1.12 


.17 


.12 


.62 


-.12 


.12 


33 


-07 

.V/ t 


06 


68 

•1X5 


- 04 


11 

.1.3 


26 


-1.01 


.18 


.22 


.70 


-.15 


.14 


.60 


-.16 


.17 




-04 


m 


27 


-.51 


-.03 


.08 


.50 


-.03 


.07 


.38 


-.01 


.09 


Al 


.12 


.11 


28 


.55 


.01 


.11 


1.22 


.36 


.27 


1.55 


.41 


.39 


.72 


-.10 


.29 


29 


-.19 


-.03 


.08 


.73 


.00 


.09 


.60 


-.02 


.11 


.69 


.07 


.13 


30 


-.41 


.03 


.07 


.48 


-.04 


.06 


.47 


-.05 


.08 


.50 


.01 


.07 


31 


-.60 


-.09 


.12 


.58 


.06 


.16 


1.19 


.11 


.25 


.59 


.04 


.14 


32 


-1.01 


.00 


.23 


.76 


-.01 


.08 


.75 


-.07 


.18 


.97 


.06 


.28 


33 


-.58 


.02 


.11 


.39 


-.01 


.09 


.35 


-.02 


.08 


.60 


-.03 


.10 


34 


-1.07 


-.17 


.22 


.20 


.11 


.05 


.42 


.08 


.12 


.59 


.04 


.17 


35 


-.85 


.02 


.06 


.63 


-.08 


.06 


.83 


-.05 


.10 


.88 


.06 


.08 


36 


-.42 


-.07 


.05 


.34 


.05 


.08 


.49 


.01 


.08 


.46 


-.04 


.14 


37 


-1.35 


.13 


.30 


.18 


-.08 


.10 


.70 


-.11 


.24 


.87 


-.02 


.25 


38 


-1.45 


-.24 


.63 


-.01 


.02 


.13 


.66 


-.01 


.30 


1.13 


.24 


.47 



(Table continues) 
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21 



Item 


A 
Q 












^3 




Mean Bias 


SD 


Mean Bias 


SD 


Mean Bias 


SD 


Mean Bias 


SD 


39 


-2.60 .14 


.56 


.27 -.16 


.11 


.83 -.20 


.30 


135 .12 


34 


40 


-.69 -.01 


.07 


.27 .01 


.11 


.37 -.05 


.14 


.90 .00 


.15 


Overall 


















Mean 


.00 -.02 


.15 


.67 -.04 


.12 


.53 -.01 


.14 


.57 .02 


.15 
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fable 3 

Means, Average Bias and Empirical Standard Errors of MIRT Parameter Estimates 



It. 




MDISC 






D 






«l 






°2 






«3 




Mean 


DlaS 




IVlCall 










SD 


Mean 


Bias 


SD 


Mean 


Bias 


SD 


1 


3.27 


-.51 


.60 


-1.34 


-.05 


.03 


.00 


.00 


.00 


90.00 


.00 


.00 


90.00 


.00 


.00 


2 


.72 


.02 


.09 


-1.86 


.05 


.22 


43.68 


-.85 


6.57 


46.33 


.85 


6.57 


90.00 


.00 


.00 


3 


.56 


-.07 


.06 


-1.38 


-.23 


.12 


28.63 


6.55 


6.83 


70.00 


.13 


8.77 


73.76 


-11.49 


11.10 


4 


1.11 


-.08 


.14 


-.99 


-.12 


.13 


36.73 


1.41 


3.74 


64.07 


-.15 


4.23 


66.20 


-1.90 


3.01 


5 


.54 


-.03 


.04 


-.95 


-.07 


.14 


35.29 


-.89 


12.32 


79.20 


-.67 


7.94 


58.37 


-.28 


13.48 


6 


.88 


-.05 


.10 


-.24 


-.04 


.15 


45.66 


4.34 


3.18 


57.74 


.75 


3.40 


61.88 


-5.94 


4.52 


7 


.90 


-.06 


.06 


-.93 


-.07 


.07 


34.08 


1.63 


3.92 


69.48 


-.51 


4.94 


64.62 


-2.02 


4.95 


8 


1.21 


-.10 


.13 


-1.09 


-.07 


.06 


15.50 


.72 


5.89 


85.55 


-2.83 


4.84 


75.90 


-.31 


5.74 


9 


1.01 


-.13 


.11 


-.85 


-.10 


.11 


35.% 


5.44 


5.94 


63.13 


-1.19 


5.69 


68.89 


-6.59 


6.40 


10 


1.19 


-.11 


.06 


-.57 


-.08 


.07 


35.44 


.81 


2.71 


72.66 


-.76 


5.19 


60.66 


-.87 


3.97 


11 


.94 


-.09 


.05 


-1.40 


-.14 


.08 


22.65 


1.38 


4.41 


79.35 


-1.17 


6.79 


71.73 


-2.33 


6.09 


12 


1.11 


.03 


.06 


-.% 


.06 


.04 


40.24 


-2.15 


4.93 


65.63 


-1.27 


3.72 


60.51 


3.41 


5.05 


13 


1.59 


-.16 


.16 


-.42 


-.07 


.05 


34.69 


-3.85 


6.89 


86.60 


-.44 


4.96 


56.10 


3.36 


8.04 


14 


1.17 


-.12 


.18 


.32 


.02 


.10 


50.81 


6.92 


6.33 


53.63 


.26 


5.24 


61.27 


-8.60 


9.55 


15 


1.07 


.11 


.19 


.47 


.10 


.13 


56.46 


5.32 


6.73 


49.48 


.64 


5.07 


59.67 


-6.87 


9.80 


16 


.88 


-.05 


.26 


.51 


.27 


.26 


71.19 


3.98 


11.46 


32.63 


-2.80 


10.84 


73.85 


-9.10 


21.01 


17 


1.74 


-.15 


.24 


-.57 


-.05 


.05 


33.82 


-4.59 


7.42 


84.48 


.83 


5.79 


57.65 


3.57 


9.34 


18 


1.02 


-.07 


.14 


.30 


.01 


.12 


50.66 


.48 


5.31 


54.20 


2.42 


7.40 


60.27 


-3.72 


5.36 


19 


.69 


-.04 


.05 


.10 


.01 


.13 


43.42 


3.65 


4.83 


59.04 


1.29 


4.79 


63.60 


-6.25 


6.41 


20 


1.31 


-.09 


.14 


.12 


-.02 


.06 


50.25 


3.74 


4.55 


52.55 


2.37 


2.96 


62.54 


-7.23 


6.58 


21 


1.33 


.14 


.15 


-.12 


.07 


.04 


53.36 


-3.82 


6.84 


45.03 


-1.46 


7.58 


70.17 


6.41 


11.37 


22 


1.19 


.22 


.11 


.09 


.06 


.08 


59.48 


.29 


4.30 


36.49 


-2.40 


3.71 


73.40 


2.33 


7.59 


23 


1.05 


.08 


.26 


.20 


.14 


.12 


67.71 


.40 


9.03 


34.41 


1.15 


5.60 


71.36 


-4.19 


7.97 


24 


1.02 


.01 


.08 


-.19 


-.02 


.06 


39.25 


-.37 


3.54 


75.55 


1.22 


3.94 


55.23 


-.50 


3.58 


25 


.99 


-.14 


.12 


1.14 


-.02 


.0^ 


50.50 


3.29 


7.40 


70.40 


2.22 


4.18 


46.41 


-4.96 


6.52 


26 


1.22 


-.21 


.28 


.83 


-.01 


.07 


54.25 


2.73 


5.64 


60.04 


4.10 


7.45 


51.21 


-6.94 


10.88 


27 


.80 


.04 


.07 


.64 


-.00 


.08 


50.52 


5.29 


5.92 


61.71 


2.55 


6.52 


53.29 


-8.04 


8.76 


28 


2.13 


.46 


.37 


-.26 


.04 


.05 


54.94 


-2.64 


5.38 


43.84 


-2.83 


5.79 


69.13 


7.10 


10.15 


29 


1.18 


.02 


.12 


.16 


.03 


.06 


51.19 


1.03 


5.34 


59.31 


1.70 


3.74 


54.34 


-2.97 


5.72 


30 


.84 


-.04 


.09 


.49 


-.01 


.06 


55.31 


1.45 


4.09 


55.97 


1.82 


3.89 


53.22 


-3.35 


3.95 


31 


1.47 


.11 


.23 


.41 


.03 


.05 


66.05 


.21 


7.13 


35.90 


-1.27 


4.32 


65.92 


.43 


4.31 


32 


1.46 


-.01 


.26 


.69 


.01 


.06 


57.22 


1.15 


6.79 


59.06 


2.91 


4.10 


48.82 


-4.05 


6.68 


33 


.80 


-.04 


.09 


.71 


.02 


.07 


60.58 


-.64 


6.85 


64.04 


.40 


6.21 


41.91 


-.52 


5.08 


34 


.75 


.10 


.15 


1.42 


.03 


.14 


74.15 


-4.91 


5.18 


57.04 


-2.11 


5.24 


37.96 


4.50 


4.60 


35 


1,37 


-.03 


.09 


.62 


.00 


.03 


62.36 


3.49 


3.26 


52.64 


1.65 


3.20 


49.99 


-4.52 


3.89 


36 


.76 


-.01 


.07 


..56 


.01 


.09 


63.20 


-4.22 


7.02 


49.4<') 


-.56 


8.87 


53.34 


3.31 


10.29 


37 


1.15 


-.11 


.29 


1.18 


-.01 


.09 


80.00 


4.77 


6.92 


52.48 


3.07 


8.66 


40.23 


-5.28 


7.33 



(Table continues) 
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L) 






«1 






«2 






«3 




It. Mean 


Bias 


SD 


Mean 


Bias 


SD 


Mean 


Bias 


SD 


Mean 


Bias 


SD 


Mean 


Bias 


SD 


38 1.33 


.18 


.51 


1.07 


.05 


.14 


89.24 


.23 


5.32 


60.04 


4.73 


10.54 


30.57 


-5.34 


10.18 


39 1.63 


-.03 


.38 


1.61 


-.06 


.11 


80.46 


5.89 


3.40 


59.19 


7.83 


8.14 


32.88 


-9i7 


7.78 


40 1.02 


-.03 


.16 


.69 


.02 


.07 


73.72 


-.01 


7.03 


68.70 


2.52 


5.96 


28.39 


-3.11 


4.68 


Overall 






























Mean 1.16 


-.03 


.17 


.01 


.00 


.09 


50.22 


1.19 


5.76 


60.33 


.63 


5.67 


59.13 


-2.56 


7.04 
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Table 4 



Average Standard Errors of NOHARM Parameter Estimates 
Using Different Anchor Items 



Anchor Items 


d 








Default 1/2 


.150 


.117 


.138 


.151 


18/24 


.170 


.211 


.237 


.221 


3/32 


.169 


.165 


.211 


.329 



o 
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Ti:blc 5 



Average Standard Errors of MIRT Parameter Estimates 
Using Different Anchor Items 



Anchor Items 


MDISC 


D 


«1 


«2 


«3 


Default 1/2 

18/24 

3/32 


.168 
.204 
.213 


.090 
.094 
.093 


5.759 
12.757 
9.142 


5.668 
12.664 

8.899 


7.042 
9.429 
14.913 



20or 



Table 6 



Means and SD's of NOHARM Paramete' Estimates 
Using Different Starting Values 



d aj aj 83 



Starting Value 


Mean 


SD 


Mean 


SD 


Mean 


SD 


Mean 


SD 


.3 


.022 


1.178 


.680 


.567 


.532 


.372 


.580 


.299 


.5' 


-.003 


1.082 


.671 


.471 


.528 


.372 


.574 


.300 


.8 


-.014 


1.026 


.674 


.419 


.516 


.357 


.562 


.286 


1.5 


-.011 


1.059 


.715 


.468 


.508 


.375 


.544 


.277 



'Default 



21 



27 



Table 7 



Correlations Between NOHARM Parameter Estimates 
Obtained Under Different Starting Values 







Starting Value 






Starting Value 




.3 


.5* 


.8 


1.5 


.3 


.5* .8 


.3 


1.000 








1.000 




.5 


.994 


1.000 






.987 


1.000 


.8 


.984 


.998 


1.000 




.957 


.991 1.000 


1.5 


.988 


.999 


.999 


1.000 


.931 


.964 .979 








^2 












Starting Value 






Starting Value 




.3 


• 

.5 


.8 


1.5 


.3 


.5* .8 


.3 


1.000 








1.000 




.5 


1.000 


1.000 






1.000 


1.000 


.8 


.986 


.990 


1.000 




.987 


.989 1.000 


1.5 


.894 


.907 


.950 


1.000 


.893 


.901 .949 



1.5 



1.000 



1.5 



1.000 



Default 
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Table 8 

Residual Analysis of NOHARM Calibration: Dataset 1 



Item 


Mean 


SD 


Skewness 


Kurtosis 


Minimum 


Maximur 


1 


-.025 


1.227 


-13.364 


266.058 


-30.113 


.974 


2 


-.003 


.955 


-.051 


.010 


-4.774 


4.075 


3 


.002 


.924 


-1.253 


1.491 


■4.365 


2.108 


4 


.009 


1.000 


6.714 


59.421 


-.998 


14.077 


5 


-.001 


.833 


8.651 


97.126 


-.980 


13.751 


6 


-.007 


.888 


1.190 


6.380 


■3.270 


8.833 


7 


.003 


.867 


-1.824 


4.545 


-6.030 


1.623 


8 


.043 


1.111 


7.311 


96.068 


-1.213 


21.981 


9 


.008 


.880 


-.720 


.693 


■3.850 


2.938 


10 


.016 


1.089 


9.411 


139.657 


-.999 


22.700 


11 


.001 


.936 


4.241 


23.170 


■1.081 


9.543 


12 


-.006 


.876 


-.384 


1.656 


-5.677 


3.379 


13 


.006 


.926 


■7.230 


71.752 


-12.605 


.815 


14 


.005 


.900 


2.245 


11.883 


-3.423 


9.445 


15 


-.008 


.957 


-.910 


2.037 


-6.226 


3.068 


16 


-.023 


1.085 


-3.167 


8.239 


-4.992 


.423 


17 


-.013 


.863 


4.356 


22.445 


-.812 


8.642 


18 


.008 


.824 


-5.585 


40.828 


-9.548 


1.063 


19 


-.002 


.940 


-.500 


.403 


-5.000 


3.657 


20 


.011 


.958 


4.259 


42.344 


■1.870 


15.423 


21 


.005 


.907 


4.035 


23.603 


-1.571 


11.014 


22 


-.007 


.903 


6.889 


103.085 


-1.206 


18.922 


23 


-.006 


.944 


-.502 


.300 


-4.450 


3.838 


24 


.010 


.961 


4.531 


49.269 


-2.093 


16.547 


25 


.002 


.892 


5.583 


46.205 


■1.307 


13.413 


26 


.002 


.899 


1.215 


2.090 


■3.758 


5.243 


27 


.000 


.903 


1.780 


6.262 


-3.947 


6.787 


28 


-.008 


.990 


-6.375 


57.331 


■14.420 


1.001 


29 


.002 


.859 


-1.657 


4.437 


-6.011 


2.223 


30 


-.004 


.%3 


7.293 


100.806 


-.882 


19.694 


31 


.009 


.684 


-8.167 


86.056 


-11.135 


.919 


32 


-.003 


.882 


4.069 


22.253 


-1.472 


8.408 


33 


.005 


.854 


-5.146 


45.846 


-13.637 


1.074 


34 


.000 


.921 


8.062 


94.821 


-1.013 


16.198 


35 


.001 


.941 


5.774 


58.480 


-1.284 


16.202 


36 


.002 


.907 


-.211 


-.345 


-3.825 


2.592 


37 


.008 


.936 


-9.337 


117.878 


-17.041 


.683 


38 


-.005 


.946 


-2.135 


6.614 


-6.827 


1.942 


39 


-.002 


.934 


-.035 


.211 


-4.452 


4.015 


40 


.004 


.782 


-1.904 


9.505 


-7.624 


2.740 


41 


.002 


.928 


2.694 


9.0'26 


-1.565 


8.373 


42 


-.007 


.960 


-1.910 


6.575 


-8.490 


2.861 



(Table continues) 
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Item 


Mean 


SD 


Skewn ess 


Kurtosis 


Minimum 


Maximum 


43 


.004 


.921 


6.014 


47.640 


-1.133 


12.848 


44 


-.002 


.887 


6.989 


65.535 


-.853 


13.188 


45 


-.001 


.916 


-2.784 


9.584 


-7.780 


1.528 


46 


-.002 


.912 


1.741 


5.055 


-2.234 


7.569 


47 


-.002 


.933 


-.567 


.330 


-4.465 


3.625 


48 


.005 


.980 


-3.258 


8.623 


-3.645 


.312 


49 


.019 


.882 


-4.253 


18.627 


-7.050 


.601 


50 


.000 


.925 


.688 


.491 


-2.656 


4.673 
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Table 9 

Residual Analysis ofNOHARM Calibration: Dataset 2 



Item 


Mean 






Kurtosis 


Minimum 


Maximum 


1 


-.024 


1.024 


-9.016 


123.787 


-20.199 


1.199 


2 


.001 


.949 


-.069 


.096 


-4.318 


4.171 


3 


-.001 


.948 


-1.147 


1.957 


-5.985 


3.654 


4 


.016 


1.168 


17.936 


518.789 


-1.208 


37.178 


5 


.009 


.803 


8.951 


106.537 


-1.229 


13.507 


6 


.000 


.894 


.794 


3.729 


-6.085 


6.176 


7 


-.003 


.878 


-2.800 


19.436 


-11.303 


2.263 


8 


.009 


1.003 


5.991 


58.410 


-1.229 


15.742 


9 


.004 


.916 


-.900 


1.227 


-5.733 


2.944 


10 


.002 


.874 


5.205 


33.354 


-1.047 


9.188 


11 


.008 


.972 


5.066 


37.213 


-1.558 


11.717 


12 


-.009 


.984 


-1.774 


21.359 


-12.502 


5.711 


13 


-.001 


.856 


-6.398 


55.508 


-11.793 


1.009 


14 


.000 


.874 


1.664 


5.563 


-3.591 


6.305 


15 


-.007 


.966 


-1.625 


12.849 


-11.905 


5.218 


16 


-.007 


.978 


-9.631 


170.501 


-23.114 


1.120 


17 


.016 


1.098 


8.449 


104.167 


-.987 


18.4% 


18 


-.008 


.897 


-10.751 


208.599 


-22.364 


1.259 


19 


.000 


.920 


-.462 


2.470 


-6.105 


3.897 


20 


.003 


.933 


3.354 


25.208 


-3.333 


13.232 


21 


.007 


.934 


4.446 


30.402 


-1.374 


11.422 


22 


.013 


1.016 


7.122 


97.734 


-1.563 


20.389 


23 


-.005 


.937 


-.690 


2.631 


-6.084 


4.491 


24 


.005 


.966 


3.087 


19.278 


-1.408 


12.789 


25 


.013 


1.032 


7.092 


76.751 


-1.367 


16.962 


26 


-.006 


.896 


.869 


1.385 


-4.621 


3.807 


27 


.004 


.926 


2.107 


10.561 


-2.872 


9.876 


28 


.003 


.877 


-4.174 


20.275 


-8.092 


1.381 


29 


.003 


.938 


-2.404 


17.081 


9.742 


5.554 


30 


.000 


.912 


4.430 


25.699 


-1.067 


10.057 


31 


-.069 


2.278 


-30.475 


7.540 


-82.177 


1.185 


32 


.002 


.933 


4.232 


27.227 


-1.359 


11.649 


33 


-.025 


1.137 


-9.691 


137.001 


-20.151 


1.042 


34 


.019 


1.096 


19.192 


544.247 


-1.290 


35.087 


35 


.008 


.976 


5.432 


40.855 


-.999 


13.574 


36 


-.005 


.937 


-.286 


2.318 


-5.871 


5.792 


37 


-.006 


.922 


-21.036 


649.789 


-30.987 


.993 


38 


-.001 


.911 


-2.178 


8.699 


-9.412 


1.670 


39 


.000 


.926 


.093 


1.565 


-5.019 


4.471 


40 


-.006 


.857 


-2.549 


17.539 


-9.660 


3.041 


41 


.016 


1.167 


9.724 


184.576 


-1.735 


27.255 


42 


.000 


.934 


-1.458 


4.051 


-6.994 


3.566 



(Table continues) 
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Item 


Mean 


SD 


Skcwncss 


Kurtosts 


Minimutn 


Maximum 


43 


.017 


1.050 


9.556 


143.616 


-1.059 


22.679 


44 


.011 


,956 


7.001 


73.232 


-1.268 


15.481 


45 


-.004 


.911 


-2.900 


13.006 


-8.299 


2.296 


46 


.004 


.920 


2.200 


9.510 


-2.710 


8.604 


47 


.006 


.931 


-.443 


1.192 


-5,463 


3.912 


48 


-.013 


1.030 


-6.099 


49.314 


-13.754 


.949 


49 


-.001 


.947 


-6.323 


53.701 


-12.397 


.832 


50 


.003 


.917 


.735 


.912 


-2.857 


5.429 
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Tabic 10 

Means and Standard Deviations of Standardized Residuals for Different Estimation Proffoms 

Program 

MIRTE TESTFACT MULTIDIM NOHARM 

Datasct Mean SD Mean SD Mean SD Mean SD 

p = 0.0 .251 1.452 .001 .893 -.026 1.321 .001 .966 

p=0.5 .253 1.312 .000 1.154 -.024 1.217 .000 .982 
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